Method for precisely and automatically positioning reference line for integrated images

ABSTRACT

The present disclosure involves a reference line determination method and system. In a process of determining a reference line, a plurality of original images containing a first spatial position information are obtained. According to the plurality of original images, a composite image containing a second spatial position information is further determined. After a composition relationship between a plurality of original images was determined, a reference line is determined on the composite image according to the spatial position information.

CROSS REFERENCE

This application is a continuation of International Application No.PCT/CN2017/075892 filed on Mar. 7, 2017, the entire contents of whichare hereby incorporated by reference.

TECHNICAL FIELD

The present disclosure generally relates to the field of medicalimaging, and more particularly, to a method and a device for preciselypositioning a reference line.

BACKGROUND

In the medical imaging field, a reference line may have greatsignificance for disease discovery, diagnosis, and treatment. Forexample, in a computed tomography (CD image of human total vertebrae, auser (e.g., a doctor), can accurately obtain a size of lesion, aposition of lesion and a position relationship between the lesion andimportant surrounding tissues based on a reference line. However,currently, display of the reference line in the medical image may haveproblems such as that the position is not accurate and no accurateadjustment can be done. For example, since the position may changeduring a composing process, the position of the reference line cannot beaccurately displayed and the reference line may not be adjusted in somemedical images (e.g., a vertebrae magnetic resonance imaging (MRI)image). Such problems may prevent a user (e.g., a doctor) fromaccurately, quickly and easily obtaining the position of the referenceline, reducing the efficiency of disease diagnosis and treatment of auser (e.g., a doctor). Therefore there is a need for a new method orsystem for positioning the reference line, so that the position of thereference line can be accurately displayed and adjusted in the medicalimage. Such a method or a system of positioning the reference line willeffectively increase the efficiency of disease diagnosis and treatmentof a user (e.g., a doctor).

SUMMARY

According to an aspect of the present disclosure, a method fordetermining a reference line for MRI, CT and/or PET images is provided.The method may include one or more steps of following operations:obtaining at least two original images, the at least two original imagescorresponding to first spatial position information; determining acomposite/integrated/fused image based on the at least two originalimages, the composite image corresponding to second spatial positioninformation; determining a composition relationship between the at leasttwo original images; and determining at least one reference line basedon the composition relationship between the at least two originalimages, the first spatial position information, and the second spatialposition information.

In some embodiments, the first spatial position information may includeat least one of position information or direction information of the atleast two original images; and the second spatial position informationmay include at least one of position information or directioninformation of the composite image.

In some embodiments, the determining a composition relationship betweenthe at least two original images may include performing at least one ofoperations to the at least two original images including: translation;rotation; zoom; and cropping.

In some embodiments, the composition relationship between the at leasttwo original images may include a registration matrix.

In some embodiments, the determining at least one reference line basedon the composition relationship between the at least two original imagesmay include: determining an intersection between the at least twooriginal images and the composite image based on the first spatialposition information and the second spatial position information;adjusting the intersection between the at least two original images andthe composite image based on the composition relationship between the atleast two original images; and determining the at least one referenceline based on the adjusted intersection between the at least twooriginal images and the composite image.

In some embodiments, the determining the intersection between the atleast two original images and the composite image based on the firstspatial position information and the second spatial position informationmay include: determining the intersection based on at least one plane ofthe at least two original images and a plane of the composite image.

In some embodiments, the determining at least one reference line mayinclude:

determining a first reference line based on the first spatial positioninformation and the second spatial position information; obtaining,based on an objective function, a transformation matrix by performing atleast one of translation or rotation operation to the first referenceline; and obtaining a second reference line by correcting the firstreference line based on the transformation matrix.

According to an aspect of the present disclosure, a reference linedetermination system for MRI, CT and/or PET images is provided. Thesystem may include a computer-readable storage medium configured tostore a plurality of executable modules and a processor, the processormay be capable of executing the plurality of executable modules storedin the computer-readable storage medium. The plurality of executablemodules may include: an image composition module and a reference linedetermination module. The image composition module may be configured to:obtain at least two original images, said at least two original imagescorresponding to first spatial position information; determine acomposite image based on the at least two original images, the compositeimage corresponding to second spatial position information; anddetermine a composition relationship between the at least two originalimages; the reference line determination module may be configured todetermine at least one reference line based on the compositionrelationship between the at least two original images, the first spatialposition information, and the second spatial position information.

In some embodiments, the first spatial position information may includeat least one of position information or direction information of the atleast two original images; and the second spatial position informationmay include at least one of position information or directioninformation of the composite image.

In some embodiments, the determining a composition relationship betweenthe at least two original images may include performing at least one ofoperations to the at least two original images including: translation;rotation; zoom; and cropping.

In some embodiments, the composition relationship between the at leasttwo original images may include a registration matrix.

In some embodiments, the determining at least one reference line basedon the composition relationship between the at least two original imagesmay include:

determining an intersection between the at least two original images andthe composite image based on the first spatial position information andthe second spatial position information; adjusting the intersectionbetween the at least two original images and the composite image basedon the composition relationship between the at least two originalimages; and determining the at least one reference line based on theadjusted intersection between the at least two original images and thecomposite image.

In some embodiments, the determining the intersection between the atleast two original images and the composite image based on the firstspatial position information and the second spatial position informationmay include: determining the intersection based on at least one plane ofthe at least two original images and a plane of the composite image.

In some embodiments, the at least one reference line may include:

determining a first reference line based on the first spatial positioninformation and the second spatial position information; obtaining,based on an objective function, a transformation matrix by performing atleast one of translation or rotation operation to the first referenceline; and obtaining a second reference line by correcting the firstreference line based on the transformation matrix.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings described herein are used to provide a furtherunderstanding of the present disclosure, all of which form a part ofthis specification. It is to be expressly understood, however, that theexemplary embodiment(s) of this disclosure are for the purpose ofillustration and description only and are not intended to limit thescope of the present disclosure. The same label in each drawingrepresents the same parts.

FIG. 1 illustrates a schematic diagram of a reference line positioningsystem according to some embodiments of the present disclosure;

FIG. 2 illustrates a schematic diagram of a processing device 130according to some embodiments of the present disclosure;

FIG. 3 illustrates a flowchart of an exemplary process for positioning areference line according to some embodiments of the present disclosure;

FIG. 4 illustrates a flowchart of an exemplary process for determining areference line based on a spatial position relationship and acomposition relationship according to some embodiments of the presentdisclosure;

FIG. 5 illustrates a flowchart of an exemplary process for correcting areference line according to some embodiments of the present disclosure;

FIG. 6 illustrates a schematic diagram of adjusting a reference lineaccording to some embodiments of the present disclosure;

FIG. 7 illustrates an architecture of a computer device which mayimplement the system disclosed in the present disclosure according tosome embodiments of the present disclosure;

FIG. 8 illustrates a structure of a mobile device which may implementthe system disclosed in the present disclosure according to someembodiments of the present disclosure; and,

FIGS. 9-A to 9-C illustrate a schematic diagram of a result of preciselypositioning of a reference line according to some embodiments of thepresent disclosure.

DETAILED DESCRIPTION

In order to illustrate the technical solutions related to theembodiments of the present disclosure, brief introduction of thedrawings referred to in the description of the embodiments is providedbelow. Obviously, drawings described below are only some examples orembodiments of the present disclosure. Those having ordinary skills inthe art, without further creative efforts, may apply the presentdisclosure to other similar scenarios according to these drawings. It isto be understood that these exemplary embodiments are given merely forthe purpose of better understanding of the present invention by thoseskilled in the art, and are not intended to limit the scope of theinvention in any way. Unless stated otherwise or obvious from thecontext, the same reference numeral in the drawings refers to the samestructure and operation.

As used in the disclosure and the appended claims, the singular forms“a,” “an,” and “The” include plural referents unless the content clearlydictates otherwise. It will be further understood that the terms“comprises,” “comprising,” “includes,” and/or “including” when used inthe disclosure, specify the presence of stated steps and elements, butdo not preclude the presence or addition of one or more other steps andelements.

Although the present disclosure makes various references to certainmodules in the system according to some embodiments of the presentdisclosure, any number of different modules may be used and run on aclient terminal and/or a server. The modules are illustrative only, anddifferent aspects of the systems and methods may use different modules.

Flowcharts are used in the present disclosure to illustrate operationsperformed by the system according to some embodiments of the presentdisclosure. It should be understood that the preceding or followingoperations may not be necessarily performed exactly in order. Instead,various steps may be processed in reverse sequence and/orsimultaneously. Moreover, other operations may also be added into theseprocedures, or one or more steps may be removed from these procedures.

FIG. 1 illustrates a schematic diagram of a reference line positioningsystem according to some embodiments of the present disclosure. Areference line may refer to an intersecting line between a first medicalimage and a second medical image displayed on the first medical image.The reference line may have important significance for various aspectssuch as description of lesion positions, determination of therapeuticplans, tumor intervention treatments, or the like, or a combinationthereof.

A reference line positioning system 100 may include one or more imagingdevices 110, one or more networks 120, one or more processing devices130 and one or more storage devices 140, or the like, or a combinationthereof.

The imaging device 110 may scan an object to be examined and obtainscanning data; the scanning data may be sent to the processing device130 via the network 120 for further process, and the scanning data alsomay be stored in the storage device 140. The object to be examined mayinclude human bodies, animals, etc. The imaging device 110 may includebut not limited to a computed tomography (CT) device, a magneticresonance imaging (MRI) device, or a positron emission computedtomography (PET) device.

The processing device 130 may process and analyze input data (forexample, scanning data and scanning images obtained by the imagingdevice 110 and/or stored in the storage device 140) to generateprocessed results. For example, the processing device 130 may generatescanning images according to the scanning data. As another example, theprocessing device 130 may segment the scanning images to obtain asegmentation result. The scanning image may be two-dimensional image orthree-dimensional image. The processing device 130 may include aprocessor and an input/output device (not shown in the figure). In someembodiments, the processor may be a server or a server group. A servergroup may be centralized, for example, a data center. A server group mayalso be distributed, for example, a distributed system. The processormay be a cloud server, a file server, a database server, a FTP server,an application server, a proxy server, a mail server, or the like, or acombination thereof. The server may be local, and may also be remote. Insome embodiments, the server may access information stored in thestorage device 140 (for example, a medical image stored in the storagedevice 140), information in the imaging device 110 (for example, amedical image photographed by the imaging device 110). In someembodiments, the input/output device may input data to the processor andmay also receive data output by the processor, and may express theoutput data in a form of a numeral, a character, an image, sound, etc.In some embodiments, the input/output device may include but not limitedto an input device, an output device, or the like, or a combinationthereof. The input device may include but not limited to a characterinput device (for example, a keyboard), an optical reading device (forexample, an optical mark reader, an optical character reader), a graphicinput device (for example, a mouse, a joystick, a light pen), an imageinput device (for example, a camera, a scanner, a fax machine), ananalog input device (for example, a language analog to digitalconversion recognition system), or the like, or a combination thereof.The output device may include but not limited to a display device, aprinting device, a plotter, an image output device, a voice outputdevice, a magnetic recording system, or the like, or a combinationthereof. In some embodiments, the processing device 130 may furtherinclude a storage device (not shown in the figure), and the storagedevice may store different information, for example, programs, data,etc. In some embodiments, intermediate data and/or a processed result(for example, a scanning image, an image segmentation result, etc.)generated by the processing device 130 may be stored in a storage deviceof the storage device 140 and/or the processing device 130, and may alsobe output by the input/output device.

In some embodiments, the data storage device 140 may refer to any devicewith a storage function in general. The data storage device 140 maymainly be used to store scanning data collected from the imaging device110 and various data generated in the operation of the processing device130. The storage device 140 or other storage devices in the system mayrefer to any media with read/write functions in general. The storagedevice 140 or other storage devices of the system may be internal orexternal to the system. The data storage device 140 may be local orremote. The data storage device 140 may include but is not limited to ahierarchical database, a networked database, a relational database, orthe like, or any combination thereof. The storage device 140 maydigitize information, and then store the digitized information in thestorage device by an electrical method, a magnetic method, an opticalmethod, or the like. The data storage device 140 may be used to storevarious types of information such as a system, software, a program, anddata. The data storage device 140 may be a device that storesinformation by electrical energy method, e.g., various memories, arandom access memory (RAM), a read-only memory (ROM), etc. The randomaccess memory may include but is not limited to a decade counting tube,a selectron, a delay line memory, a Williams tube, a dynamic randomaccess memory (DRAM), a static random access memory (SRAM), a thyristorrandom access memory (T-RAM), a zero-capacitor random access memory(Z-RAM), or the like, or any combination thereof. The read only memorymay include but is not limited to a bubble memory, a twistor memory, afilm memory, a plated wire memory, a magnetic-core memory, a drummemory, a CD-ROM, a hard disk, a tape, a NVRAM, a phase-change memory, amagneto-resistive random access memory, a ferroelectric random accessmemory, a nonvolatile SRAM, a flash memory, an electrically erasableprogrammable read-only memory (EEPROM), an erasable programmableread-only memory, a mask read only memory, a floating connected gaterandom access memory, a nano random access memory, a racetrack memory, aresistive random access memory, a programmable metallization unit, orthe like, or any combination thereof. The storage device 140 may be adevice that stores information by magnetic energy method, e.g., a harddisk, a soft disk, a tape, a magnetic core storage, a bubble memory, aU-Disk, a flash memory, etc. The storage device 140 may be a device thatstores information by an optical method, e.g., a CD, a DVD, etc. Thestorage device 140 may be a device that stores information by themagneto-optical method, e.g., a magneto-optical disk, etc. Access modesof the storage device 140 may include random access mode, serial accessmode, read-only access mode, or the like, or any combination thereof.The storage device 140 may be a non-permanent memory or a permanentmemory. The storage devices described above is only examples. Thedatabase used in the reference line positioning system 100 are notintended to be limiting.

The network 120 may be a single network or a combination of multiplenetworks. The network 120 may include but is not limited to a local areanetwork, a wide area network, a public network, a dedicated network, awireless local area network, a virtual network, a metropolitan areanetwork, a public switched telephone network, or the like, or anycombination thereof. The network 120 may include a variety of networkaccess points, such as wired or wireless access points, a base station,or network switching points. A data source may be connected to thenetwork 120 through the access points. Information may be sent via thenetwork.

It should be noted that the above description of the system is merelyprovided for the purpose of illustration, and not intended to limit thescope of the present disclosure. For persons having ordinary skills inthe art, modules may be combined in various ways, or connected withother modules as sub-systems, and various modifications andtransformations in form and detail may be conducted under the teachingof the present disclosure. However, those modifications andtransformations may not depart from the spirit and scope of thisdisclosure. For example, the storage device 140 may be a cloud computingplatform with data storage function including but not limited to apublic cloud platform, a private cloud platform, a community cloudplatform, a hybrid cloud platform, etc. All such transformations arewithin the protection scope of the present disclosure.

FIG. 2 illustrates a schematic diagram of a processing device 130according to some embodiments of the present disclosure. The processingdevice 130 may include an image composition module 210, a reference linedetermination module 220, and a reference line correction module 230.The modules described in FIG. 2 may be implemented by a computerdescribed in FIG. 7 via a CPU unit 720. The modules may be directly(and/or indirectly) connected. Apparently, the processing device 130 inFIG. 2 may only represent some embodiments of the present disclosure,for those of ordinary skill in the art, modification, addition ordeletion may be made according to FIG. 2 without any creative works. Forexample, two modules therein may be combined into one module,alternatively, one module may be segmented into two or more modules.

In some embodiments, the image composition module 210 may generate oneor more composite images based on original images. The image compositionmodule 210 may obtain the original images from the imaging device 110.The obtained original images may include but not limited to one or moremedical images (for example, a CT image, an MRI image, a PET image,etc.). The original images may be two-dimensional images orthree-dimensional images. The original images may include but notlimited to raw images and/or processed images. The raw images may referto images directly obtained according to the scanning data (for example,medical images). The processed images may refer to images obtained byprocessing the raw images. The processing of the raw images may includebut not limited to image enhancement, image reconstruction,three-dimensional reconstruction, image filtering, image coding (forexample, compression coding), image format conversion, image rendering,image zoom, or the like, or a combination thereof.

The number of the obtained original images may be two or more. In someembodiments, several original images may be generated by scanning thesame object to be examined (for example, a human body, a part of thehuman body, etc.) from different angles. In some embodiments, theimaging device 110 may generate a plurality of image segments based on aplurality of different angles. Each image segment may include at leastone original image, and the original image may have related or similarspatial position information. For example, the imaging device 110 mayphotograph the human total vertebrae from three different angles toobtain three image segments: an image segment A (such as, a cervicalvertebra image segment), an image segment B (such as, a thoracicvertebra image segment), and an image segment C (such as, a lumbarvertebra image segment). The number of original images contained in theimage segment A, the image segment B, and the image segment C may be thesame or different.

The original images may be two-dimensional images or three-dimensionalimages. For example, the original images may be two-dimensional imagesshowing the human total vertebrae. As another example, the originalimages may be three-dimensional images showing a human liver. In someembodiments, the original images may be generated based on originalimages with a same modality or original images with differentmodalities. For example, the original images may be generated based onmedical images (e.g., an MRI image, a CT image, a PET image, etc.) withdifferent modalities. The medical images with a same modality (medicalimages with different modalities) may correspond to a same object to beexamined or different objects to be examined and/or different parts ofthe same object to be examined. For example, a composite image of thehuman total vertebrae may be composite based on an MRI image (e.g., acervical vertebra image segment) and a CT image (e.g., a lumbar vertebraimage segment).

In some embodiments, the reference line determination module 220 maydetermine a reference line. The reference line may be used to display apositional relationship between another image and current referenceimage on a reference image. The reference line may include elements suchas one or more straight lines, a line segment, a curved line, a point,etc. The reference line may include any number of pixels. In someembodiments, the reference line may be commonly expressed using astraight line, a dashed line, a line segment, etc., on the referenceimage. The reference image may be a composite image or an originalimage. For example, during a head CT scan, the imaging device 110 maygenerate an axial image, a sagittal image, or a coronal image of acranium. The reference line determination module 220 may determineand/or display one or more reference lines in the axial image, and thereference line may be used to show a positional relationship between thecoronal image and the axial image (as shown in FIG. 9A). The axial imagemay represent an image transecting the object to be examined into upperand lower parts when the object to be examined (for example, a humanbody) stands on the ground, the image being perpendicular to thesagittal image and the coronal image. The sagittal image may representan image slitting the object to be examined into left and right partsbased on a front-rear direction of the object to be examined when theobject to be examined (for example, a human body) stands on the ground.The coronal image may represent an image cropping the object to beexamined into front and rear parts based on a left-right direction ofthe object to be examined when the object to be examined (for example, ahuman body) stands on the ground.

In some embodiments, the reference line determination module 220 maydetermine and display a plurality of reference lines between theoriginal image and the current composite image on the composite image.For example, on the composite image of a human total vertebrae, thereference line determination module 220 may display five referencelines. The reference lines may show a positional relationship betweenthe composite image of the human total vertebrae and five axial images(e.g., five lumbar vertebra axial images). A user (e.g., a doctor) mayanalyze an accurate position of an axial image of a certain lumbarvertebra on the human total vertebrae (e.g., the axial image of afracture lumbar vertebra may belong to the second lumbar vertebra) toperform disease diagnosis and subsequent treatment.

In some embodiments, the reference line correction module 230 maycorrect a position of the reference line. In some embodiments, thereference line determination module 220 may display a reference linecorresponding to an original image on the composite image. The positionsof the reference lines corresponding to the original images may beinaccurate since a position relationship between the original images maychange during a generation process of the composite image. For example,a position of the reference line displayed on the composite image of thehuman total vertebrae may be beyond a upper and lower range of avertebrae (for example, the position of the reference line is higherthan altlas), or an angle of the reference line and vertebrae may notcorrespond to an actual scanning angle (for example, a direction of thereference line of the vertebral axial image is parallel or substantiallyparallel to a vertebral direction), etc. The reference line correctionmodule 230 may manually or automatically correct the reference lineposition.

In some embodiments, a user (e.g., a doctor) may input a correctioncommand via an input/output component 760 or an input/output (I/O) unit850. The correction command may include translation, rotation, addition,deletion of the reference line, or the like, or a combination thereof.The reference line correction module 230 may obtain the correctioncommand and correct the reference line accordingly. In some embodiments,a user (e.g., a doctor) may preset reference line correction rules (forexample, parallel spacing of the reference line is greater than 5 mm orother threshold, or the specific thickness of the reference line, or thespecific length of the reference line, etc.) based on the network 120, acomputer 700 or a mobile device 800. According to the reference linecorrection rules, the reference line correction module 230 mayautomatically correct the reference line accordingly.

FIG. 3 illustrates a flowchart of an exemplary process for positioning areference line according to some embodiments of the present disclosure.A process 300 may be implemented by one or more hardware, software,firmware, or the like, or a combination thereof. In some embodiments,the process 300 may be implemented by one or more processing devices(for example, the processing device 130 shown in FIG. 1) and/or acomputing device (for example, a computer shown in FIGS. 7 and 8) foroperating the image composition module 210.

At 310, the image composition module 210 may obtain at least twooriginal images. The original images may be scanning images. Thescanning images may include but not limited to CT images, MRI images, orPET images. The scanning images may be two-dimensional images orthree-dimensional images. The scanning image may include but not limitedto raw images and/or processed images. The raw images may refer toimages obtained directly according to the scanning data. The processingimages may refer to images obtained by processing the raw images. Theprocessing of the raw images may include but not limited to imageenhancement, image reconstruction, three-dimensional reconstruction,image filtering, image coding (for example, compression coding), imageformat conversion, image rendering, image zoom, or the like, or acombination thereof.

The image enhancement may mean increased contrast of whole or partialregions of images. For example, in a human vertebral MRI image, contrastof vertebrae and surrounding nerves or soft tissues may be enhanced, soas to allow an imaging technician or a doctor to quickly and easilyidentify boundaries of vertebral edges. As another example, for acraniocerebral MRI image, certain lesions (epileptic focus) or braintissues of important functional areas may be added to facilitate thedoctor to determine a surgically cropping range, thereby reducingdamages to normal brain tissues, particularly, brain tissues of theimportant functional areas, while resecting the lesions within thelargest range. In some embodiments, the image enhancement may includecontrast enhancement, noise removal, background removal, edgesharpening, filtering, wavelet transform, or the like, or a combinationthereof.

Image reconstruction may represent generating images in any planaraccording to existing MRI scanning images. Three-dimensionalreconstruction may represent obtaining hepatic three-dimensional imagesaccording to hepatic two-dimensional scanning images. For example, theimage composition module 210 may allow a vertebral MRI image to beconverted into a visualization toolkit image format (VTI format) from adigital imaging and communications in medicine (DICOM) format.

Image encoding may also refer to image compression, representingexpressing image or information contained in the image using a smallnumber of bits when certain image quality (for example, SNR) isachieved. Image rendering may represent converting high dimensionalinformation into low dimensional information, for example, convertingthree-dimensional information into two-dimensional information.

In some embodiments, the image composition module 210 may obtainscanning data from the imaging device 110, and reconstruct the rawimages based on the scanning data. Methods for MRI reconstruction by theimage composition module 210 may include an MRI reconstruction methodbased on K-space filling or an MRI reconstruction method based on imagedomain. The MRI reconstruction method based on K-space filling mayinclude a half-Fourier imaging method, a SMASH imaging method, or anAUTO-SMASH imaging method, etc. The MRI reconstruction method based onimage domain may perform MRI image reconstruction by using some prioriinformation on an MRI image, the method may decrease data scanning time,thereby accelerating a process of MRI imaging. The MRI reconstructionmethod based on the image domain may reconstruct images according todifferent sensitivity information between coils, or may reconstructimages according to sparsity of the MRI image (for example, MRI imagereconstruction may be performed by using a compression sensing method.)

At 320, the image composition module 210 may obtain spatial positioninformation of the original images. The spatial position information mayinclude three-dimensional coordinate information, two-dimensionalcoordinate information of the original images, spatial positioninformation in a specific image format, etc. The spatial positioninformation of the original images may be obtained from image data inany image formats. The image formats may include but not limited to aDICOM format, a VTI format, etc. The image data may include one or morelogic levels. The image data may include a file header and a data setgroup in a physical structure. The file header may include a preamble(for example, 128 bytes) and a prefix (for example, 4 bytes). Thepreamble may have a fixed structure or not. For example, in someembodiments, when there is no content, bytes may be set to OOH and theprefix may be a DICM character string, so as to identify a DICOM file.The data set may have stored all necessary information for operating theDICOM file, and may include a plurality of data elements. Each dataelement may include four fields of Tag, Vale Representation, ValueLength, and Value Field, and have stored a formatting rule and contentof the element information.

The spatial position information of the original images obtained by theimage composition module 210 may include position information of theoriginal images (for example, image position (patient) in DICOM),direction information of the original images (for example, imageorientation (patient) in DICOM), and other information related tospatial positions. The position information of the original images mayinclude coordinate information related to the original images (forexample, coordinates of pixels in the first row and the first column ofthe original images (for example, upper left corner)). The coordinateinformation may include one or more coordinates in any coordinatesystem. In some embodiments, the position information of the originalimages may include one or more coordinates in a specific coordinatesystem. For example, the spatial position information of the originalimages may be represented as O1 (x1, y1, z1).

The direction information of the original images may include anyinformation related to directions in the original images. For example,the direction information of the original images may include directioninformation (one or more directions, vectors representing directions,triangle function values corresponding to one or more directions, etc.)corresponding to one or more parts (for example, pixels in the first rowand/or pixels in the first column) of the original images. For example,the direction information of the original images may include a directionvector in the first row, and a direction vector in the first column inthe original images. As another example, the direction information ofthe original images may be represented as a normal vector of theoriginal images. In some embodiments, the normal vector may be across-product of the direction vector in the first row and the directionvector in the first column.

In some embodiments, the original image may be represented as Equation(1):

(x−x1)nx+(y−y1)ny+(z−z1)nz=0  (1)

wherein, (x1, y1, z1) is the position information of the originalimages, and (nx, ny, nz) is the direction information of the originalimages (e.g., the normal vector).

At 330, the image composition module 210 may determine a composite imagebased on the at least two original images. The original images mayinclude a first original image, a second original image, etc. The imagecomposition module 210 may determine a composite image by performingtransformation such as translation, rotation, scale up, shear, etc. onthe original images. For example, the image composition module 210 mayset any one of the at least two original images as a fixed referenceimage (e.g., the first original image), and translation and/or rotationtransformation may be performed on the other original images accordingto the fixed reference image.

At 330, the image composition module 210 may obtain spatial positioninformation of the composite image. The spatial position information mayinclude three-dimensional coordinate information of the composite image,two-dimensional coordinate information of the composite image, spatialposition information in a specific image format, etc.

The spatial position information obtained by the image compositionmodule 210 may include position information of the composite image (forexample, image position patient in DICOM), direction information of thecomposite image (for example, image orientation patient in DICOM), etc.The position information of the composite image may include coordinateinformation related to the composite image (for example, coordinates ofpixels in the first row and the first column (the upper left corner) inthe composite image). The coordinate information may include one or morecoordinates in any coordinate system. In some embodiments, the positioninformation of the composite image may include one or more coordinatesin specific coordinate systems. For example, the spatial positioninformation of the original images may be represented as O2 (x2, y2,z2).

The direction information of the composite image may include anyinformation related to directions in the composite image. For example,the direction information of the composite image may include directioninformation (one or more directions, vectors representing directions,triangle function values corresponding to one or more directions, etc.)corresponding to one or more parts (for example, pixels in the first rowand/or pixels in the first column) of the composite image. For example,the direction information of the composite image may include a directionvector in the first row and a direction vector in the first column inthe composite image. As another example, the direction information ofthe composite image may be represented as the normal vector of theoriginal images. In some embodiments, the normal vector may be across-product of the direction vector in the first row and the directionvector in the first column.

In some embodiments, the spatial position information of the compositeimage may be represented as Equation (2):

(x−x2)Nx+(y−y2)Ny+(z−z2)Nz=0  (2)

wherein, (x2, y2, z2) is the position information of the compositeimage, and (Nx, Ny, Nz) is the direction information of the compositeimage.

In some embodiments, the image composition module 210 may determine thecomposite image based on a composition algorithm. The compositionalgorithm may include an area-based composition algorithm, and afeature-based composition algorithm. The area-based compositionalgorithm may calculate differences in gray values of the originalimages using the least squares method, determine a similarity ofoverlapping areas between the original images based on the differencesin the gray values, determine a range and a position of the overlappingareas between the original images based on the similarity, and finally,generate the composite image. The feature-based composition algorithmmay include feature extraction and feature alignment. Feature alignmentalgorithms may include algorithms such as cross correlation, distancetransformation, dynamic programming, structure matching, chain codecorrelation, etc.

At 340, the reference line determination module 220 may determine acomposition relationship between the original images (for example, afirst original image and a second original image). The compositionrelationship may be used to represent position relationships between theoriginal images in a process of determining a composite image. Theposition relationships between the original images may includetranslation, rotation, scale change, shear, or the like, or acombination thereof. In some embodiments, during CT scanning of a humantotal vertebrae, the image composition module 210 may select a firstoriginal image as a fixed reference image, and compose images based on acomposition algorithm. For example, the image composition module 210 mayadjust a second original image with respect to a position of the fixedreference image (the first original image), and may also adjust a size,a direction, etc. of the second original image. During thetransformation, the reference line determination module 220 maydetermine and/or store the position relationship between the originalimages (e.g., between the first original image and the second originalimage). The position relationship between the original images may berepresented as a registration matrix. In some embodiments, theregistration matrix may be a 4*4 matrix:

$\begin{pmatrix}a_{11} & a_{12} & a_{13} & a_{14} \\a_{21} & a_{22} & a_{23} & a_{24} \\a_{31} & a_{32} & a_{33} & a_{34} \\a_{41} & a_{42} & a_{43} & a_{44}\end{pmatrix},$

wherein, first three rows of the registration matrix may representaffine transformation, including rotation, translation, and shear.Fourth row of the matrix may represent projective transformation.

At 350, the reference line determination module 220 may determine areference line based on spatial relative position information and thecomposition relationship between the original images. In someembodiments, the reference line determination module 220 may determine areference line directly based on spatial position information of thefirst composite image, spatial position information of the secondcomposite image and composition relationship between the secondcomposite image and the first composite image. For example, thereference line determination module 220 may directly determine areference line on the first composite image based on the positioninformation of the first composite image, the position information ofthe second composite image and the registration matrix (e.g., a 4*4matrix) between the first original image and the second original image,and the reference line may correspond to the second composite image.

In some embodiments, the reference line determination module 220 maydetermine an initial reference line on the composite image based on thespatial position information of the original images and the spatialposition information of the composite image; and determine a correctedreference line based on the initial reference line and the compositionrelationship. For example, on a CT image of a human total vertebrae, thereference line determination module 220 may determine an initialreference line based on a three-dimensional coordinate (e.g.,three-dimensional coordinates of upper-left pixels) of the originalimages and spatial position information of a composite image; andfurther determine a position of the corrected reference line or displaythe corrected reference line based on the initial reference line and aregistration matrix between the original images.

FIG. 4 illustrates a flowchart of an exemplary process for determining areference line based on a spatial position relationship and acomposition relationship according to some embodiments of the presentdisclosure. A process 400 may be implemented by one or more hardware,software, firmware, or the like, or a combination thereof. In someembodiments, the process 400 may be implemented by one or moreprocessing devices (for example, the processing device 130 shown inFIG. 1) and/or a computing device (for example, a computer shown inFIGS. 7 and 8) for operating the image composition module 210.

At 410, the reference line determination module 220 may determine anintersection between the original image and the composite image. In someembodiments, the reference line determination module 220 may determinean intersection between the original image and the composite image basedon spatial information of original image and spatial information of thecomposite image. The intersection between the original image and thecomposite image may be determined based on a plane of the compositeimage and the original image located or an equation of a plane thereof.In some embodiments, the reference line determination module 220 maydetermine a plane of the composite image or an equation of a planethereof based on the spatial position information of the composite image(e.g., position information or direction information). The referenceline determination module 220 may determine a plane of the originalimage or an equation of a plane thereof based on the spatial positioninformation of the original image (e.g., position information ordirection information). The equation of the plane of the composite imageand/or the equation of the plane of the original image may include anintercept equation of a plane, a point normal equation of a plane, ageneral equation of a plane, a normal equation of a plane, or the like,or a combination thereof. For example, the reference line determinationmodule 220 may obtain a pixel coordinate O1 (x1, y1, z1) based on theposition information of the original image; and may obtain a normalvector (dx, dy, dz) of the original image based on the directioninformation of the original image. The reference line determinationmodule 220 may also determine a point normal form plane equation basedon the pixel coordinate O1 (x1, y1, z1) and the normal vector (dx, dy,dz).

In some embodiments, the intersection between the original image and thecomposite image may be determined based on face to face intersecting ora solution of two equations of planes. In some embodiments, thereference line correction module 230 may calculate a solution betweenthe equation of the plane of the original image and the equation of theplane of the composite image, to determine the intersection between theoriginal image and the composite image. In some embodiments, if there isno intersection between the original image and the composite image, thereference line correction module 230 may determine that the originalimage and the composite image are in parallel or process in other ways.

At 420, the reference line correction module 230 may adjust theintersection based on the composition relationship between the originalimages. In some embodiments, the reference line correction module 230may adjust the intersection based on a registration relationship betweenthe original images, and obtain an adjusted intersection. For example,the composition relationship between the first original image (a fixedreference image) and the second original image may be represented by aregistration matrix (e.g., a 4*4 matrix) or may be stored in the storagedevice 140. The reference line correction module 230 may calculate asolution of a plane equation corresponding to the second original imageand a plane equation corresponding to the first composite image, andfurther determine an intersection between the first original image andthe second image. The intersection of the second original image may beadjusted on the first composite image based on the registration matrix(e.g., a 4*4 matrix). As another example, if the second original imagetranslates five units towards the positive direction of a X axis withrespect to the first original image (a fixed reference image) during thecomposing process, the reference line correction module 230 may, afterdetermining an intersection between the composite image and the secondoriginal image, translate the intersection five units towards thenegative direction composite image and obtain an adjusted intersection.

At 430, the reference line correction module 230 may determine aposition of the reference line based on the adjusted intersection. Insome embodiments, the intersections between the original image and thecomposite image may be more than one (for example, two). For example,the reference line correction module 230 may determine two intersectionson two relatively parallel sides in a plane of the composite image,based on four sides in a plane of the original image and a plane of thecomposite image. The two intersections may be adjusted in 420, andfurther two adjusted intersections may be determined. Based on the twoadjusted intersections of the composite image, the reference linecorrection module 230 may connect the two adjusted intersections, andgenerate a line segment. The line segment may be used as a referenceline of the original image corresponding to the composite image.

In some embodiments, the intersections between the original image andthe composite image may be two or more. For example, the reference linecorrection module 230 may determine 100 intersections on the compositeimage based on the plane equation of the original image and the planeequation of the composite image; after the 100 intersections have beenadjusted, the reference line correction module 230 may retain part ofthe adjusted intersections on the composite image, or may delete part ofthe adjusted intersections. The deletion of part of the adjustedintersections may be based on a length, a region, etc., of the referenceline displayed on the composite image. For example, if the length of thereference line is set to 5 cm by the system 100, the reference linecorrection module 230 may delete intersections within 5 cm, anddetermine a line segment as a reference line based on the retainedintersections.

FIG. 5 illustrates a flowchart of an exemplary process for correcting areference line according to some embodiments of the present disclosure.A process 500 may be implemented by one or more hardware, software,firmware, or the like, or a combination thereof. In some embodiments,the process 500 may be implemented by one or more processing devices(for example, the processing device 130 shown in FIG. 1) and/or acomputing device (for example, a computer shown in FIGS. 7 and 8) foroperating the image composition module 210.

At 510, the reference line correction module 230 may obtain at least twooriginal images, and determine a composite image based on the at leasttwo original images. The original images may be scanning images. Thescanning images may include but not limited to CT images, MRI images orPET images. The scanning images may be two-dimensional images orthree-dimensional images.

In some embodiments, the original images may include a plurality ofimage segments, each image segment may include at least one originalimage, and the original image may have the same or similar spatialposition information. For example, the imaging device 110 may photographthe human total vertebrae based on three different angles and obtainthree image segments: an image segment A (e.g., a cervical vertebraimage segment), an image segment B (e.g., a thoracic vertebra imagesegment), and an image segment C (e.g., a lumbar vertebra imagesegment); and the image segment A, the image segment B and the imagesegment C may include 50, 40, and 55 original images, respectively.

In the process of generating the composite image, the image compositionmodule 210 may determine a composition relationship using the imagesegment as a unit. For example, the CT image of the human totalvertebrae may include three image segments: an image segment A, an imagesegment B and an image segment C; each image segment may include atleast one original image, and the original image in the same imagesegment may have similar spatial position information, for example, thesame plane direction or the same scanning mode (for example, the imagesegment A may be axial scanning, the image segment B may be sagittalscanning, or the image segment C may be coronal scanning); and in thecomposing process, the image composition module 210 may take the imagesegment A as a fixed reference image segment, and perform transformationsuch as translation, rotation, scale change, or shear on the imagesegment B or the image segment C, so as to obtain the composite image.

At 520, the reference line determination module 220 may determine and/ordisplay a first reference line, based on a spatial position relationshipof the original image and a spatial position relationship of thecomposite image. The first reference line may be an initial referenceline, or may be a reference line adjusted based on the initial referenceline and the composition relationship. In some embodiments, thereference line determination module 220 may directly determine a planeof the original image and a plane of the composite image based on thespatial position relationship of the original image and the spatialposition relationship of the composite image, and determine anintersection between the two planes; the reference line determinationmodule 220 may directly determine a line segment or a straight linebased on the intersection, as the first reference line. For example, thereference line determination module 220 may determine a first referenceline of an axial image of the fourth thoracic vertebra on the compositeimage of the human total vertebrae based on a plane of the axial imageof the fourth thoracic vertebra and a plane of the composite image ofthe human total vertebrae, and the first reference line may pass thefourth thoracic vertebra on the composite image and be perpendicular toor substantially perpendicular to a human total vertebrae direction.

In some embodiments, the reference line determination module 220 maydetermine the plane of the original image and the plane of the compositeimage based on the spatial position relationship of the original imageand the spatial position relationship of the composite image, and maydetermine an intersection between the determined planes and an initialreference line. The reference line determination module 220 may adjust aposition of the initial reference line based on the compositionrelationship, and determine a position of the first reference line basedon the adjusted initial reference line. For example, the reference linedetermination module 220 may adjust an initial reference line betweenthe plane of the original image and the plane of the composite imagebased on a registration matrix (e.g., a 4*4 matrix); and may determine afirst reference line based on the adjusted initial reference line.

At 530, the reference line correction module 230 may perform acorrection operation to the first reference line based on an objectivefunction, and obtain an adjustment matrix. The correction operation mayinclude translation, rotation, scale change, shear, or the like, or acombination thereof. The correction may be performed on the firstreference line. The transformation matrix may represent the correctionoperation of the first reference line. In some embodiments, thetransformation matrix may be stored in the storage device 140 or thenetwork 120. In some embodiments, in the three dimensional space, if theplane of the original image and the plane of the composite image may beexpressed in homogeneous coordinates, the transformation matrixcorresponding to the first reference line may include a 4*4 matrix:

$\begin{pmatrix}a_{11} & a_{12} & a_{13} & a_{14} \\a_{21} & a_{22} & a_{23} & a_{24} \\a_{31} & a_{32} & a_{33} & a_{34} \\a_{41} & a_{42} & a_{43} & a_{44}\end{pmatrix},{wherein},\begin{pmatrix}a_{11} & a_{12} & a_{13} & a_{14} \\a_{21} & a_{22} & a_{23} & a_{24} \\a_{31} & a_{32} & a_{33} & a_{34}\end{pmatrix}$

may represent rotation, translation, or shear, and (α₄₁ α₄₂ α₄₃ α₄₄) mayrepresent projective transformation.

At 540, the reference line correction module 230 may correct the firstreference line based on the transformation matrix, and obtain a secondreference line. In some embodiments, the reference line correctionmodule 230 may determine a second reference line based on atransformation matrix, and the second reference line may be representedas Equation (3):

Y′=mY  (3)

wherein, Y′ represents a second reference line; Y represents a firstreference line without a correction operation; and m represents atransformation matrix in the three dimensional space, and m may be a 4*4matrix.

In some embodiments, the second reference line may be displayed on thecomputer 700 or on the mobile device 800 via the network 120. Take theMM scanning image of the human total vertebrae as an example, theprocessing device 130 may display the composite image and the secondreference line on the mobile device 800 (e.g., a smartphone) of a user(e.g., a doctor) via the network 120, allowing the user to performremote diagnosis and treatment; taking the human head CT scanning imageas another example, the processing device 130 may send the compositeimage and the second reference line to a cloud; a user (e.g., a doctor)may obtain the composite image and the second reference line via thecloud and display the composite image and the second reference line onthe smartphone, so that the user can remotely receive imaginginformation.

FIG. 6 illustrates a schematic diagram of adjusting a reference lineaccording to some embodiments of the present disclosure. A process 600may be implemented by one or more hardware, software, firmware, etc., orthe combination thereof. In some embodiments, the process 600 may beimplemented by one or more processing devices (for example, theprocessing device 130 shown in FIG. 1) and/or a computing device (forexample, a computer shown in FIGS. 7 and 8) operating the imagecomposition module 210.

At 610, the processing device 130 may obtain at least two MRI images(for example, a first MRI image, a second MRI image, etc.). The at leasttwo MRI images may involve any scanned object. For example, the at leasttwo MRI images may include an MRI scanning image of a human totalvertebrae, an MRI scanning image of a head, etc. In some embodiments,the MRI scanning image of the human total vertebrae may include at leastthree image segments: a cervical vertebra image segment, a thoracicvertebra image segment, and a lumbar vertebra image segment. The imagesegments may include one or more original MRI images, and the MRI imagewithin each image segment may have the same or similar compositionrelationship in a subsequent composing process. In some embodiments, theat least two MRI images may be generated at different time. For example,the at least two MRI images may include a vertebrae MRI image beforesurgery and a vertebrae MRI image after the surgery.

At 620, the processing device 130 may obtain spatial positioninformation of each MM image. The spatial position information mayinclude three-dimensional coordinate information, two-dimensionalcoordinate information of the original images, or spatial positioninformation in a specific image format. In some embodiments, spatialposition information of the composite image obtained by the imagecomposition module 210 may include position information (for example,image position patient in a DICOM format), direction information (forexample, image orientation patient in a DICOM format), etc. In someembodiments, the processing device 130 may obtain spatial positioninformation of different image segments. For example, on the MRIscanning image of the human total vertebrae, the processing device 130may obtain spatial position information of a cervical vertebra imagesegment, a thoracic vertebra image segment, and a lumbar vertebra imagesegment, so as to determine position changes of each segment, aftercomposite, with respect to a certain fixed reference segment.

At 630, the processing device 130 may determine a composite image basedon a composition algorithm. The composition algorithm may include acomposition algorithm based on areas and a composition algorithm basedon features. The composition algorithm based on areas may calculate,based on gray values of the original images, a difference of the grayvalues using the least squares method; judge similarity of overlappingareas between the original images based on the difference of the grayvalues; and determine a range and a position of the overlapping areasbetween the original images based on the similarity, finally, generatinga composite image. The composition algorithm based on features mayinclude feature extraction and feature alignment. Feature alignmentalgorithms may include algorithms e.g. a cross correlation algorithm atransformation algorithm a dynamic programming algorithm, a structurematching algorithm, a chain code correlation algorithm, etc. Forexample, using the composition algorithm based on areas, the processingdevice 130 may generate a composite image based on at least two MRIimages or a plurality of image segments. The composite image may be acoronal image of the human total vertebrae.

At 640, the processing device 130 may determine a registration matrixbetween the MRI images based on the composite image. In someembodiments, the processing device 130 may take a certain MRI image orimage segment (e.g., a cervical vertebra image segment) as a fixedreference, and then determine a composition relationship of other MRIimages or image segments (e.g., a thoracic vertebra image segment or alumbar vertebra image segment). For example, the processing device 130may take a first MRI image as a fixed reference. The compositionrelationship may be represented as a registration matrix. In someembodiments, in a composing process of three-dimensional space, theprocessing device 130 may determine a plane of each MRI image beforecombination and a plane of that after combination, and determine apartial registration matrix based on position relationships of each MRIimage with respect to the fixed reference image. In some embodiments, inthree-dimensional space, if a plane of the MRI image is represented byhomogeneous coordinates, after a composition operation, a correspondingregistration matrix may be represented as a 4*4 matrix, e.g. Equation(4):

P′=nP  (4)

wherein, P′ represents an MRI image after combination; P represents anMRI image without a composition; and n represents a registration matrixin a three-dimensional space, and n is a 4*4 matrix.

At 650, the processing device 130 may determine a first reference linebased on the registration matrix and a spatial position relationship.The processing device 130 may determine a first reference line of an MRIimage on a composite image of the total vertebrae. In some embodiments,the processing device 130 may determine an equation of a plane of an MRIimage and an equation of a plane of a total vertebrae composite image;and based on the equation of the plane of the MRI image and the equationof the plane of the total vertebrae composite image, the processingdevice 130 may determine an intersection between the MRI image and thecomposite image of the total vertebrae. For example, based on spatialposition information (e.g., position information and/or directioninformation) of a second MRI image in the cervical vertebra imagesegment, a first equation of a plane may be determined; based on spatialposition information of the total vertebrae composite image, a secondequation of a plane may be determined; and based on the first equationof the plane and the second equation of the plane, the processing device130 may determine the intersection between the second MRI image in thecervical vertebra image segment and the total composite image. In someembodiments, based on the intersection, the processing device 130 maydirectly determine the first reference line on the composite image ofthe total vertebrae. In some embodiments, based on the intersection anda registration matrix, the processing device 130 may further adjust theposition of the intersection, and determine a first intersecting line.

At 660, the processing device 130 may obtain a transformation matrix ofthe first reference line. In some embodiments, a user (e.g., a doctor)may manually adjust the position of the first reference line. The manualtransformation may include translation, rotation, scale change, shear,etc., of the first reference line. A user (e.g., a doctor) may input amanual transformation command via an input/output component 760 in thecomputer 700. The processing device 130 may accept the manualtransformation command, and effectuate the manual transformation of theposition of the first reference line. In some embodiments, the manualtransformation may also include deletion, addition, formatting, or thelike, or a combination thereof, of the first reference line. Forexample, a doctor may input a manual transformation command via theinput/output component 760, to delete one or more first reference lines.In some embodiments, the processing device 130 may automatically adjustthe position of the first reference line based on user setting. Forexample, the processing device 130 may adjust the position of the firstreference line on the composite image based on an objective function setby a user. In some embodiments, the processing device 130 may determinea transformation matrix based on transformation of the position of thefirst reference line. The transformation matrix may be a 4*4 matrix. The4*4 matrix may be stored in the storage device 140 or in a cloud via thenetwork 120.

At 670, the processing device 130 may update the position of the firstreference line based on the transformation matrix, and may generate asecond reference line. In some embodiments, a user may select whether ornot to make a reference line update. For example, the user may select toupdate the first reference line via the input/output component 760, theprocessing device 130 may send a transformation matrix to the computer700 or the mobile device 800; and based on the transformation matrix,the processing device 130 may update the position of the first referenceline, and may generate the second reference line, the second referenceline may display the position relationship between the MRI image and thecomposite image of the total vertebrae more precisely, so that a user(e.g., a doctor) may conveniently perform disease diagnosis andtreatment.

FIG. 7 illustrates a structure of a computer device which may implementthe system disclosed in the present disclosure according to someembodiments of the present disclosure. The system in the presentembodiment may use a functional block diagram to explain a hardwareplatform containing a user interface. The computer may be ageneral-purpose computer or a specific-purpose computer. Both computersmay be configured to implement the system in this embodiment. Thecomputer 700 may be used to implement any components described currentlywhich may offer information required by image integration. For example,the processing device 130 may be implemented by the computer such as acomputer 700 through its hardware devices, software programs, firmware,or the like, or any combination thereof. For convenience, only onecomputer is depicted in FIG. 7, but the related computer functionsdescribed in this embodiment to provide the required information for theimage integration may be implemented by a set of similar platforms in adistributed mode, which may decentralize a processing load of thesystem.

The computer 700 may include a communication port 750 that may beconnected with a network to implement data communication. The computer700 may also include a central processing unit (CPU) that may alsoinclude one or more processors to conduct program instructions. Theexemplary computer platform may include an internal communication bus710, a program storage unit in different forms, and a data storage unit.For example, a hard disk 770, a read only memory (ROM) 730, arandom-access memory (RAM) 740, various data files that may beconfigured to be used for computer processing and/or communication, andpossible program instructions executed by the CPU. The computer 700 mayalso include an input/output component 760 that supports input/outputdata streams between the computer and other components (e.g., the userinterface 780). The computer 700 may also receive programs and data viaa communication network.

According to some embodiments of the present disclosure, FIG. 8illustrates a structure of a mobile device which may implement thesystem disclosed in the present disclosure. In this embodiment, a mobiledevice 2800 for displaying and interacting with location-relatedinformation may include but is not limited to a smartphone, a tabletcomputer, a music player, a portable gaming machine, a globalpositioning system (GPS) receiver, a wearable computing device (e.g.,eyeglasses, watches, etc.), or other forms that may be found indescriptions elsewhere of the present disclosure. The mobile device 800in this embodiment may include one or more central processing units(CPUs) 840, graphical processing units (GPUs) 830, a display 820, amemory 860, an antenna 810 (e.g., a wireless communication unit), astorage unit 890, and one or more input/output (I/O) devices 850. Anyother suitable components, the components may include but not limitedto, system buses or controllers (not shown in the figure) may also beincluded in the mobile device 800. As shown in FIG. 8, a mobileoperating system 870 (e.g. iOS, Android, Windows Phone, etc.) and one ormore applications 880 may be loaded into the memory 860 from the storageunit 890 and may be executed by the central processing unit 840. Theapplications 880 may include a browser or other mobile applications thatare suitable for receiving and processing information relating to thereference line on the mobile device 800. The information relating to thereference line with respect to users may be obtained by the input/outputsystem devices 850 and may be provided to the processing device 130,and/or other components of the system 100, e.g., via the network 120.

In order to implement the different modules, units, and their functionsas described in the previous disclosure, a computer hardware platformmay be used as a hardware platform (e.g., the processing device 130,and/or other components of the system 100) for one or more of theelements described above. The hardware elements, operating systems, andprogramming languages of such computers are common in nature. It isassumed that those skilled in the art are familiar with these techniquesand may use the techniques described herein to provide the requiredinformation for image integration. A computer containing user interfaceelements may be used as a personal computer (PC) or other types ofworkstations or terminal devices, and may also be used as servers afterbeing properly programmed. It may be appreciated that those skilled inthe art will be familiar with such structures, procedures, and generaloperations of such computer equipment, and therefore no additionalexplanation is required for all figures.

FIGS. 9-A through 9-C are schematic diagrams illustrating how toprecisely position reference lines according to some embodiments of thepresent disclosure. FIG. 9-A is an axial image of a head; FIG. 9-B is asagittal image of the head; and the FIG. 9-C is a coronal image of thehead. In FIG. 9-A, 910 may be a reference line determined by theprocessing device 130 based on the axial image of the head and thecoronal image of the head, and the reference line displays anintersecting line of the coronal image of the head (as shown in FIG.9-C) on the axial image of the head. In FIG. 9-B, 920 may be a referenceline determined by the processing device 130 based on the sagittal imageof the head and the coronal image of the head, and the reference linedisplays an intersecting line of the head coronal image (as shown inFIG. 9-C) on the sagittal image of the head.

The above descriptions may disclose different aspects of methods forproviding information required on image integration for implanting otheroperations by procedures. The procedures in the disclosure may beconsidered as “product” or “merchandise” existing in the form ofexecutable codes and/or related data, which may be participated orimplemented by a computer readable medium. A tangible and permanentstorage medium may include any memory or storage used by a computer, aprocessor, a similar device, or a related module, for example, asemiconductor memory, a tape drive, a disk drive, or other devices thatmay provide storage functions for software at any time.

All software or a part of it may communicate over a network, such as theInternet or other communication networks. Such communications may loadsoftware from one computer device or processor to another. For example:from a management server or host computer in an imaging system loaded toa computer hardware platform, or other computer environment, or thesimilar function system relating to provide information required by theon-demand services. Thus, another medium that may deliver softwareelements may also be used as physical connections between local devices,such as light waves, radio waves, electromagnetic waves, etc., throughcables, cables, or air. A physical medium used for a carrier, such as acable, a wireless connection, or an optical cable, may also beconsidered a medium for carrying software. Here, unless the physicalstorage medium is restricted, other terms that may represent thecomputer or machine “readable medium” represent the medium in which aprocessor executes any instruction.

Therefore, a computer readable medium may have many forms, including, avisible storage media, a carrier media, or a physical transmissionmedia. Stable storage media may include compact disks (CD), disks, orstorage systems used in other computers or similar devices that mayenable the system components described in the diagrams. Unstable storagemedia may include a dynamic memory, such as the main memory of acomputer platform. The tangible transmission medium may include acoaxial cable, a copper cable, an optical fiber, and a circuitry formingthe bus within the computer system. A carrier transmission medium maytransmit electrical signals, electromagnetic signals, acoustic signals,or light wave signals, which may be generated by radio frequency orinfrared data communication. A computer readable medium may include ahard disk, a floppy disk, a magnetic tape, any other magnetic medium; aCD-ROM, a DVD, a DVD-ROM, any other optical media; a hole punched card,and any other physical storage medium containing a hole patterns; a RAM,a PROM, an EPROM, a FLASH-EPROM, or any other memory chip tape; acarrier for data or instructions transmission, cable, a connectiondevice for carrier transmission, or any other computer that may be usedto read the code and/or data. In the form of these computer readablemedia, there are a variety of processes that occur when the processor isexecuting instructions and delivering one or more results.

The contents disclosed in this application can be diversified andimproved. For example, different system components described above maybe implemented by hardware, or software. For example, the system may beinstalled on the existing server. In addition, the location informationdisclosed here may be implemented through a firmware, a combination offirmware and software, a combination of firmware and hardware, or acombination of hardware, firmware and software.

The above descriptions may describe this application and/or some otherexamples. According to the above contents, the application may also makedifferent variations. The topics disclosed in this application may beimplemented in different forms and examples, and this application may beapplied to a large number of applications. All the applications,modifications and changes required in the post claims are within thescope of this application.

What is claimed is:
 1. A method for determining a reference line,comprising: obtaining at least two original images, the at least twooriginal images corresponding to first spatial position information;determining a composite image based on the at least two original images,the composite image corresponding to second spatial positioninformation; determining a composition relationship between the at leasttwo original images; and determining at least one reference line basedon the composition relationship between the at least two originalimages, the first spatial position information, and the second spatialposition information.
 2. The method of claim 1, wherein the at least twooriginal images include at least one of a CT image, an MRI image, or aPET image.
 3. The method of claim 1, wherein the first spatial positioninformation includes at least one of position information and directioninformation of the at least two original images; and the second spatialposition information includes at least one of position information anddirection information of the composite image.
 4. The method of claim 1,wherein the determining a composition relationship between the at leasttwo original images comprises performing at least one of operations tothe at least two original images including: translation; rotation; scalechange; and shear.
 5. The method of claim 1, wherein the compositionrelationship between the at least two original images comprises aregistration matrix.
 6. The method of claim 1, wherein the determiningat least one reference line based on the composition relationshipbetween the at least two original images comprises: determining anintersection between the at least two original images and the compositeimage based on the first spatial position information and the secondspatial position information; adjusting the intersection between the atleast two original images and the composite image based on thecomposition relationship between the at least two original images; anddetermining the at least one reference line based on the adjustedintersection between the at least two original images and the compositeimage.
 7. The method of claim 6, wherein the determining theintersection between the at least two original images and the compositeimage based on the first spatial position information and the secondspatial position information comprises: determining the intersectionbased on at least one plane of the at least two original images and aplane of the composite image.
 8. The method of claim 1, wherein thedetermining at least one reference line comprises: determining a firstreference line based on the first spatial position information and thesecond spatial position information; obtaining, based on an objectivefunction, a transformation matrix by performing at least one oftranslation or rotation operation to the first reference line; andobtaining a second reference line by correcting the first reference linebased on the transformation matrix.
 9. A reference line determinationsystem comprises: a computer-readable storage medium configured to storea plurality of executable modules, the plurality of executable modulescomprising: an image composition module configured to: obtain at leasttwo original images, said at least two original images corresponding tofirst spatial position information; determine a composite image based onthe at least two original images, the composite image corresponding tosecond spatial position information; determine a compositionrelationship between the at least two original images; and a referenceline determination module configured to determine at least one referenceline based on the composition relationship between the at least twooriginal images, the first spatial position information, and the secondspatial position information; and a processor capable of executing theplurality of executable modules stored in the computer readable storagemedium.
 10. The reference line determination system of claim 9, whereinthe at least two original images include at least one of a CT image, anMRI image, or a PET image.
 11. The reference line determination systemof claim 9, wherein the first spatial position information includes atleast one of position information and direction information of the atleast two original images; and the second spatial position informationincludes at least one of position information or direction informationof the composite image.
 12. The reference line determination system ofclaim 9, wherein the determining a composition relationship between theat least two original images comprises performing at least one ofoperations to the at least two original images including: translation;rotation; scale change; and shear.
 13. The reference line determinationsystem of claim 9, wherein the composition relationship between the atleast two original images comprises a registration matrix.
 14. Thereference line determination system of claim 9, wherein the determiningat least one reference line based on the composition relationshipbetween the at least two original images comprises: determining anintersection between the at least two original images and the compositeimage based on the first spatial position information and the secondspatial position information; adjusting the intersection between the atleast two origin 0.1 images and the composite image based on thecomposition relationship between the at least two original images; anddetermining the at least one reference line based on the adjustedintersection between the at least two original images and the compositeimage.
 15. The reference line determination system of claim 9, whereinthe determining the intersection between the at least two originalimages and the composite image based on the first spatial positioninformation and the second spatial position information comprises:determining the intersection based on at least one plane of the at leasttwo original images and a plane of the composite image.
 16. Thereference line determination system of claim 9, wherein the determiningat least one reference line comprises: determining a first referenceline based on the first spatial position information and the secondspatial position information; obtaining, based on an objective function,a transformation matrix by performing at least one of translation androtation operation to the first reference line; and obtaining a secondreference line by correcting the first reference line based on thetransformation matrix.